Meta-Analysis of Multiple Simulation-Based Experiments

Computer simulations facilitate the study of real world phenomena by providing safe, controlled, flexible, and repeatable experimental environments at costs far lower than other options. Both the validity of the simulation model and the scope of its validity determine the degree to which the findings can be generalized and applied to real world situations and problems. In practice, no single model captures all of the important aspects of a phenomena of interest, nor is applicable over a wide set of missions and circumstances. Thus, effectively utilizing a variety of models in a prospective meta-analysis (a set of common hypotheses and controllable variables and comparable metrics) offers the opportunity to improve validity and extend the findings to a broader range of real world situations. NATO SAS-085, a research group exploring C2 Agility and Requisite Maturity, conceived and developed an international meta-analysis approach for studying various aspects related to C2 Agility from multiple simulation-based experiments. This paper presents the methodology they employed which was inspired by the prospective meta-analysis domain. The challenges that arose from differences among these experiments, differences in the ways C2 Approaches were instantiated, and differences in the measures of success and in the conditions they considered are discussed. Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE JUN 2013 2. REPORT TYPE 3. DATES COVERED 00-00-2013 to 00-00-2013 4. TITLE AND SUBTITLE Meta-Analysis of Multiple Simulation-Based Experiments 5a. CONTRACT NUMBER